Gradient-flow adaptive importance sampling for Bayesian leave one out cross-validation with application to sigmoidal classification models

Chang JC, Li X, Xu S, Yao HR, Porcino J, Chow CC (2024). ArXiv [Preprint] 2402.08151v2. PMID: 38711425; PMCID: PMC11071546. https://arxiv.org/abs/2402.08151

Overview

Leave-one-out cross-validation (LOO-CV) is the gold standard for Bayesian model comparison, but it is computationally expensive because it requires refitting the model for every data point. Pareto-smoothed importance sampling (PSIS-LOO) approximates LOO-CV from a single posterior fit, but it can fail when individual observations are highly influential — a common situation in classification models with sigmoidal link functions.

This paper introduces a gradient-flow adaptive importance sampling method that addresses these failures. Rather than discarding problematic observations or falling back to expensive refitting, the approach uses gradient flows to continuously adapt the importance distribution toward each leave-one-out posterior. The method is particularly well suited to sigmoidal classification models where observations near the decision boundary exert strong influence on the posterior. The result is a reliable and efficient approach to Bayesian model comparison that works where standard importance sampling breaks down.